Defects the first real-data run exposed, and the fix that unblocks ingest - #651

Three defects the first real-data run exposed, each one invisible to a green CI. Every fix is proven to fail without it.

They share a root cause worth naming: fixtures and guards were built from docstrings and from the code's own output, so nothing compared them against what production actually writes.

Read the commanded counts 2-BM actually writes (713171618c)

_scalar_int handled only 0-dimensional datasets. 2-BM writes /process/acquisition/rotation/num_angles as a 1-element int32 array, NumPy 2 refuses int() on those, and the reader turned the TypeError into None. So every commanded count was None at the beamline and the shortfall check had nothing to compare against, which also makes DATA-8 unanswerable from the record.

_scalar_str already handled this exact shape, which is why the timestamp parsed and the integers did not. The scalar shape is now a fixture parameter rather than an assumption, so the existing matrix runs against both forms.

Refuse to export a record whose payloads are strings (3a07f5da0f)

asyncpg returns jsonb as text unless the connection registers the codecs postgres.pool installs. Nothing checked. The first export of the live 2-BM database produced a bundle whose every payload was one opaque string, wrote it, hashed it, and passed verify_record_hash.py — perfectly self-consistent about the wrong structure, which is the one failure a record meant to be verifiable by a stranger must not have. Only redaction noticed, by crashing on a str where it wanted a mapping.

Key the redaction table on what events are stored under (ae39e5299e)

The table is looked up by events.event_type but was keyed on the class name, with a comment asserting those are the same. The class is ActorRegistered; the append path writes "ActorRegisteredV2". Redaction refuses an unknown event type and every database holds an Actor registration from bootstrap, so the published-export path could not succeed against any real record. A legacy record would have exported fine, since the V1 string was the one in the table.

The drift test compares the generator against its own committed output, so a generator deriving the wrong key gives a wrong table and a green test. The generator now asks each module's own event_type_name, and the new fitness test reaches the same answer by a deliberately different route (static literal scan, no imports, no generator), because a guard sharing the generator's mechanism shares its blind spot.

One behaviour change, stated rather than buried: the V1 string is no longer in the table, so a pre-vault database carrying legacy ActorRegistered rows now fails closed on export. That is the safe direction, since a V1 payload carries the name field the PII vault removed, but retired wire names have no home in a generated table yet.

Let a deployment say which timestamp is the acquisition time (b1c51735df)

2-BM's scan files carry two timestamps and the obvious one is wrong. Measured across six consecutive files, every file's start_date equals the previous file's end_date, while every end_date falls within five seconds of its own close. The areaDetector timestamp attribute refreshes only while frames flow, so a file opened for scan N carries the value left from scan N-1's last frame. The timestamp PV itself is healthy; read live it returns the correct instant.

The reader only ever read start_date, and the ingest policy is that a parseable file timestamp beats an operator's, with both supplied being refused as ambiguous. Together those would have recorded a scan we watched from end to end as captured the previous evening, uncorrectably. So ingest at 2-BM stayed blocked rather than writing a false fact into the record.

Which timestamp to believe is a deployment fact, not a layout fact. The reader is now told, and reads what it is told:

• Default unchanged (start_date), so no other deployment moves.
• No fallback: declaring end_date and getting a file without one is a refusal, since silently reading the other timestamp returns exactly the value the declaration exists to avoid.
• Description.start_date becomes captured_at (the field no longer always holds a start date) and gains captured_at_source, so the record states which fact it used.
• The policy itself is deliberately unchanged: a file value still beats an operator's, because the fix for a bad writer is to declare the good timestamp, not to let callers assert over the file.

The descriptor records the choice for humans, the setting drives the runtime, and they agree by hand until the descriptor is read at runtime, which the descriptor docstring says out loud.

Naming reviewed by naming-r3-reviewer: captured_at_source, the scan_ prefix, and the descriptor key were confirmed against R1-R6, and its one finding (make the lookup constant private) is applied.

Not in this PR

The retrospective gate review of #650 raised further P0s on the published-artifact path (manifest not bound to the written record, raw Run UUIDs in the published manifest, tier-2 clearances describing a different producer than the Conductor, is_simulated inverted). Those are separate slices and will carry the full panel.

🤖 Generated with Claude Code